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A  vision  sensing  system  for  the  measurement  of  auto-guidance  pass-to-pass  and  long-term  errors  was 
implemented  to  test  the  steering  performance  of  tractors  equipped  with  auto-guidance  systems.  The 
developed  test  system  consisted  of  an  optical  machine  vision  sensor  rigidly  mounted  on  the  rear  of  the 
tested  tractor.  The  center  of  the  drawbar  hitch  pin  point  was  used  as  the  reference  from  which  to  mea¬ 
sure  the  deviation  of  the  tractor’s  actual  travel  path  from  its  desired  path.  The  system  was  built  and 
calibrated  to  a  measurement  accuracy  of  better  than  2  mm.  To  evaluate  the  sensor,  two  auto-guidance 
systems  equipped  with  RTK-level  GNSS  receivers  were  tested  and  the  results  for  different  travel  speeds 
compared.  Pass-to-pass  and  long-term  errors  were  calculated  using  the  relative  positions  of  a  reference  at 
a  collocated  point  when  the  tractor  was  operated  in  opposite  directions  within  15  min  and  more  than  1  h 
apart,  respectively.  In  addition  to  variations  in  speed,  two  different  auto-guidance  steering  stabilization 
distances  allowed  for  comparison  of  two  different  definitions  of  steady-state  operation  of  the  system. 
For  the  analysis,  non-parametric  cumulative  distributions  were  generated  to  determine  error  values  that 
corresponded  to  95%  of  the  cumulative  distribution.  Both  auto-guidance  systems  provided  95%  cumu¬ 
lative  error  estimates  comparable  to  51  mm  (2  in.)  claims  and  even  smaller  during  Test  A.  Higher  travel 
speeds  (especially  5.0  m/s)  significantly  increased  measured  auto-guidance  error,  but  no  significant  dif¬ 
ference  was  observed  between  pass-to-pass  and  long-term  error  estimates.  The  vision  sensor  testing 
system  could  be  used  as  a  means  to  implement  the  auto-guidance  test  standard  under  development  by 
the  International  Standard  Organization  (ISO).  Third-party  evaluation  of  auto-guidance  performance  will 
increase  consumer  awareness  of  the  potential  performance  of  products  provided  by  a  variety  of  vendors. 

©  2010  Elsevier  B.V.  All  rights  reserved. 


1.  Introduction 

Auto-guidance  (also  called  auto-steering)  systems  represent  a 
rapidly  expanding  technology  in  precision  agriculture  that  is  based 
on  the  use  of  global  navigation  satellite  system  (GNSS)  receivers  to 
perform  field  operations  in  a  strict  geometrical  relationship  with  a 
previous  travel  path  or  other  predefined  geographical  coordinates, 
without  direct  inputs  from  an  operator.  Although  auto-guidance 
systems  available  to  producers  have  different  levels  of  operation 
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accuracy  as  well  as  sensor  configurations  and  interfaces,  their 
performance  is  frequently  associated  with  an  anticipated  level  of 
auto-guidance  error,  usually  referred  to  as  cross-track  error  (XTE). 
This  error  can  be  attributed  to  numerous  uncertainties,  includ¬ 
ing:  (1)  geographic  positioning  errors;  (2)  vehicle  dynamics;  (3) 
the  implement  tracking  behind  the  vehicle;  (4)  the  field  environ¬ 
ment  (slopes,  soil  condition,  etc.).  Manufacturers  of  auto-guidance 
systems  publish  claims  that  rely  on  a  variety  of  different  test  proce¬ 
dures,  and  as  a  result,  consumers  cannot  use  marketing  information 
to  compare  the  performance  of  different  products.  Therefore,  there 
is  a  need  to  develop  a  standardized  procedure  to  test  and  report  the 
performance  of  GNSS-based  auto-guidance  systems. 

The  first  step  in  testing  GNSS-based  equipment  involves  eval¬ 
uation  of  the  static  performance  of  GNSS  receivers  by  placing  the 
antenna  in  a  fixed  georeferenced  location  (ION,  1997)  and  logging 
measurements  made  by  the  receiver.  Agricultural  operations  are 
dynamic  in  nature;  therefore,  tests  of  GNSS  receivers  used  in  agri¬ 
culture  should  be  performed  while  in  motion.  Stombaugh  et  al. 
(2002)  and  later  Stombaugh  et  al.  (2008)  provide  general  guide¬ 
lines  for  a  dynamic  test.  Two  main  dynamic  GNSS  receiver  testing 
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methods  were  defined:  (1)  fixture-based  testing,  which  involves 
mounting  the  GNSS  receiver  on  a  platform  that  is  operated  along 
a  fixed  path  with  known  geographic  coordinates  and  (2)  vehicle- 
based  testing,  in  which  the  tested  set  of  receivers  is  placed  on  the 
top  of  a  vehicle  along  with  a  superior  performance  measurement 
system,  possibly  a  real-time  kinematic  (RTK)  GNSS  receiver.  Advan¬ 
tages  of  fixture-based  testing  include  the  ability  to  calculate  errors 
with  respect  to  the  actual  (surveyed)  geographic  coordinates  and 
the  repeatability  of  the  testing  procedure.  The  advantage  of  vehicle- 
based  testing  is  it  can  represent  actual  field  operations. 

Though  fixture-based  testing  cannot  be  used  to  evaluate  the 
actual  performance  of  a  vehicle  operated  using  a  GNSS  receiver,  Han 
et  al.  (2004)  implemented  a  vehicle-based  approach  to  test  eight 
commercially  available  DGPS  receivers  from  four  different  manu¬ 
facturers  with  five  alternative  differential  correction  services.  All 
eight  tested  GPS  receivers  were  mounted  simultaneously  on  a  test 
vehicle  at  least  1  m  apart  from  each  other  to  reduce  possible  signal 
interference.  An  RTK-level  GPS  receiver  was  mounted  in  the  center 
of  the  test  platform  to  provide  the  vehicle  reference  positions.  The 
vehicle  was  manually  driven  on  a  travel  path  as  straight  as  possible 
in  the  north-south  direction,  with  each  test  consisting  of  six  parallel 
passes  approximately  305  m  (1000  ft)  long.  The  desired  pass-to- 
pass  spacing  was  6.1 0  m  (20  ft).  Off-track  errors  were  determined  as 
the  root  mean  squared  difference  between  the  horizontal  position 
determined  by  the  reference  receiver  (with  the  appropriate  offset 
compensations)  and  the  tested  receivers.  Pass-to-pass  error  was 
defined  as  the  difference  between  corresponding  off-track  errors.  It 
was  noted  that  travel  speed  might  play  an  important  role  in  quan¬ 
tifying  receiver  accuracy  since  at  lower  speeds,  the  pass-to-pass 
average  errors  tended  to  be  larger.  Han  et  al.  (2004)  associated  the 
increase  in  pass-to-pass  error  with  the  longer  time  needed  to  com¬ 
plete  the  test  course  when  moving  slower.  However,  due  to  data 
limitations,  complete  analysis  of  the  effect  of  travel  speed  was  not 
conducted. 

A  similar  approach  can  be  used  to  test  the  performance  of  nav¬ 
igation  aids,  known  as  light  bar  systems.  Light  bar  systems  assist 
the  operator  of  an  agricultural  vehicle  in  steering  it  according  to 
GNSS  position  estimates.  Buick  and  Lange  (1998)  and  later  Buick 
and  White  (1999)  compared  the  efficiencies  of  foam  marker  and 
GPS-based  light  bar  guidance  systems.  Field  efficiencies  were  deter¬ 
mined  by  measuring  the  actual  areas  of  skips  and  overlaps  for 
different  ground  speeds  and  offline  distances  (based  on  vehicle 
track  records).  In  another  study,  Ehsani  et  al.  (2002)  tested  differ¬ 
ent  GPS-based  light  bar  systems  by  mounting  them  on  the  roof  of 
a  tractor  and  driving  nine  swaths  parallel  to  a  pre-set  A-B  line.  In 
both  cases,  an  RTK  receiver  was  used  to  determine  the  actual  travel 
path. 

The  testing  of  auto-guidance  systems  has  become  the  latest 
challenge  when  it  comes  to  the  GNSS-based  operation  of  agricul¬ 
tural  vehicles.  The  measurement  system  for  test  instrumentation 
must  have  at  least  ten  times  greater  accuracy  than  the  system 
being  tested  (ION,  1997).  This  means  that  for  auto-guidance  sys¬ 
tems  equipped  with  meter  and  decimeter-level  GNSS  receivers,  a 
centimeter-level  sensor,  such  as  an  RTK-level  GNSS  receiver,  can 
be  used.  However,  since  many  advanced  auto-guidance  options 
employ  centimeter-level  GNSS  receivers,  an  appropriate  test  sys¬ 
tem  should  be  capable  of  making  millimeter-level  measurements. 

Harbuck  et  al.  (2006)  employed  optical  surveying  equipment 
to  track  vehicle  motion  without  the  involvement  of  GNSS-based 
equipment.  A  rugged  360-degree  tracking  prism  was  mounted  to 
the  towing  hitch  on  the  rear  of  the  tractor.  Position  data  was 
recorded  using  a  total  station  equipped  with  a  special  function 
that  made  it  possible  to  follow  the  moving  prism  by  the  use  of 
servo  motors  in  the  total  station  base.  During  each  test,  the  trac¬ 
tor  was  operated  through  a  straight  pass  using  the  auto-guidance 
system,  and  the  relative  position  of  the  tractor  hitch  was  contin¬ 


uously  recorded.  The  claimed  5-mm  measurement  error  of  the 
total  station  was  applicable  under  ideal  conditions,  but  this  error 
increased  to  20  m  during  the  test.  Consequently,  the  order  of  mag¬ 
nitude  required  for  greater  accuracy  by  the  measurement  system 
was  no  longer  valid. 

Adamchuk  et  al.  (2007)  developed  a  linear  potentiometer  array 
that  measured  the  horizontal  position  of  a  reference  cart  perpen¬ 
dicular  to  the  direction  of  travel  as  it  repeatedly  passed  over  a 
series  of  stationary  metal  triggers  installed  on  the  surface  of  the 
pavement  used  for  testing.  The  system  had  an  approximate  res¬ 
olution  of  20  mm  and  did  not  rely  on  a  GNSS  signal.  Although 
both  methods  are  suitable  for  many  non-RTK-based  options,  testing 
auto-guidance  systems  with  a  claimed  accuracy  of  around  20  mm 
would  require  a  more  precise  solution. 

The  objectives  of  this  research  were  (1)  to  develop  instru¬ 
mentation  and  test  methodology  for  measuring  relative  XTE  with 
millimeter-level  accuracy;  (2)  to  evaluate  the  method  developed 
by  comparing  the  performance  of  tractors  with  auto-guidance  sys¬ 
tems  operated  at  various  travel  speeds;  (3)  to  recommend  a  test 
procedure  for  measuring  pass-to-pass  and  long-term  relative  XTE 
in  a  repeatable  manner. 

2.  Materials  and  methods 

2.1.  Instrumentation  development 

Testing  auto-guidance  systems  requires  a  method  of  measure¬ 
ment  that  is  accurate  enough,  yet  easy  to  use  and  adaptable  to 
multiple  situations.  After  a  number  of  options  involving  different 
optical  referencing  techniques  were  considered,  the  final  choice 
was  the  machine  vision  approach.  Various  machine  vision  sen¬ 
sors  are  used  extensively  in  industry  for  real-time  monitoring  of 
product  dimensions  and  quality  control.  Following  a  test  concept 
pursued  by  Adamchuk  et  al.  (2007),  the  vision  sensor  was  mounted 
on  the  tested  vehicle  to  monitor  a  permanent  reference  line  on  the 
pavement  below.  As  the  vehicle  moved  along  the  test  track,  it  was 
possible  to  measure  the  relative  position  of  the  tested  vehicle  with 
respect  to  the  permanent  reference  line  in  every  location  along  the 
test  track. 

Since  the  test  was  focused  on  auto-guidance  systems  with  RTK- 
level  GNSS  receivers,  a  1 .2-m  field  of  view  was  assumed  appropriate 
so  that  the  line  remained  visible  to  the  sensor  during  the  entire 
test.  To  achieve  the  2-mm  sensor  resolution  required  by  the  20-mm 
claimed  accuracy  would  involve  a  600-pixel  array  ( 1 200  mm/2  mm) 
in  the  horizontal  direction  (perpendicular  to  the  direction  of  travel). 
A  Cognex  In-Sight®  DVT  545  high-speed  vision  sensor  with  an 
internal  processor  (Cognex  Corporation,  Natick,  MA)4  and  a  NAV 
LFC-9F1B  9-mm  lens  was  considered  sufficient.  The  sensor  had 
a  640  x  1048  pixel  array  with  a  26°  field  of  view.  This  provided 
approximately  1.2-mm  resolution  at  the  testing  surface  when 
mounted  1.5  m  above  the  ground  and  pointed  downward.  The  sen¬ 
sor  was  capable  of  automatically  adjusting  exposure  and  aperture 
settings  for  varying  lighting  conditions  and  could  process  images  at 
the  rate  of  approximately  30  frames/s.  The  vision  sensor  calibration, 
cross-track  position  measurements,  and  other  adjustments  were 
made  using  Intellect™  (Cognex  Corp.,  Natick,  MA)  software. 

Relative  position  measurements  performed  with  the  vision  sen¬ 
sor  were  synchronized  with  geographic  locations  to  allow  the 
matching  of  measurements  obtained  during  different  passes.  An 
additional  GNSS  receiver  was  used  to  obtain  geographic  longitude 


4  Mention  of  a  trade  name,  proprietary  product,  or  company  name  is  for  presen¬ 
tation  clarity  and  does  not  imply  endorsement  by  the  authors  or  the  University  of 
Nebraska-Lincoln,  nor  does  it  imply  exclusion  of  other  products  that  may  also  be 
suitable. 
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30  5  m  (100  ft)  121  9m  (400  ft)  305m(100ft) 

Fig.  1.  Nebraska  Tractor  Test  Laboratory  track  with  inner  and  outer  marks. 


and  latitude,  time,  and  GNSS  signal  quality  for  further  data  process¬ 
ing.  Data  acquisition  was  accomplished  using  a  specially  developed 
Lab  VIEW®  (National  Instruments,  Inc.,  Austin,  TX)  interface. 

2.2.  Test  procedure  development 

The  test  procedure  developed  was  based  on  a  typical  field  oper¬ 
ation  in  which  a  series  of  back  and  forth  parallel  passes  across  a 
certain  distance  are  performed.  At  the  end  of  each  pass,  the  vehicle 
is  turned  around  and  returns  on  a  path  adjacent  to  the  previous 
pass  with  an  offset  equal  to  the  fixed  width  of  the  implement 
(swath  width).  In  this  case,  relative  XTE  can  be  defined  as  the  differ¬ 
ence  between  the  desired  and  actual  swath  widths.  If  the  distance 
between  two  passes  is  less  than  the  swath  width,  an  overlap  will 
occur;  a  distance  greater  than  the  swath  width  produces  a  skip. 
Pass-to-pass  error  of  auto-guidance  is  defined  as  the  relative  XTE 
between  two  consecutive  passes  that  occur  within  a  15-min  time 
interval.  Long-term  auto-guidance  error  is  defined  as  the  relative 
XTE  between  two  consecutive  passes  that  occur  more  than  1  h  apart 
with  dissimilar  GNSS  satellite  configurations  in  the  sky. 

To  accommodate  these  definitions,  each  test  consisted  of  three 
test  runs  with  three  passes  about  7.5  min  long  made  in  alternating 
directions.  An  appropriate  test  location  should  have  a  pavement 
surface  that  would  not  change  over  time  and  could  be  replicated  in 
various  geographic  areas.  Since  tractor  performance  testing  is  typ¬ 
ically  done  on  concrete  pavement,  the  same  approach  was  used  in 
developing  an  auto-guidance  system  test  procedure.  The  concrete 
tractor  test  track  of  the  Nebraska  Tractor  Test  Laboratory  (NTTL, 
Lincoln,  NE)  was  selected  to  also  serve  as  the  test  track  for  the 
auto-guidance  systems  testing  (Fig.  1). 

The  track  consists  of  two  east- west  oriented  straight  passes  sep¬ 
arated  by  39.9  m  (131  ft).  Both  passes  are  relatively  level,  with  the 
total  length  of  the  central  line  around  the  track  being  approximately 
615  m  (2018  ft).  Each  straight  pass  of  the  track  was  6.7  m  (22  ft) 
wide  with  an  expansion  seam  in  the  middle,  which  was  designated 
as  the  permanent  reference  line.  Theoretically,  the  east-west  direc¬ 
tion  of  the  track  presented  the  most  challenging  auto-guidance 


operation  environment  from  the  viewpoint  of  GNSS  performance, 
as  the  positioning  error  associated  with  latitude  is  typically  greater 
than  the  longitudinal  error  for  this  location. 

To  adapt  the  ideal  (back  and  forth)  field  operation  pattern  to  the 
geometry  of  the  test  track,  the  test  trial  consisted  of  a  sequence  of 
counterclockwise,  clockwise  and  counterclockwise  laps  around  the 
track  as  shown  in  Fig.  2. 

The  initial  A-B  line  was  established  along  the  northern  pass  and 
the  auto-guidance  equipment  was  set  with  a  39.9-m  swath  width. 
The  test  tractor  was  operated  in  auto-guidance  mode  along  each 
of  the  two  passes.  During  each  pass,  the  relative  location  of  the 
tractor’s  representative  vehicle  point  (RVP)  with  respect  to  the  ref¬ 
erence  line  was  measured  using  the  visual  sensor.  For  each  location 
around  the  track,  the  difference  between  these  relative  position 
measurements  (adjusted  for  the  direction  of  travel)  was  used  to 
define  relative  XTE. 

For  the  test  vehicle,  the  decision  was  made  to  use  the  most 
common  platform  on  which  auto-guidance  systems  are  installed. 
Mechanical  front  wheel  assist  tractors  with  dual  rear  tires  in  the 
range  of  PTO  power  from  110  to  220  kW  (150  to  300  hp)  were 
selected.  The  drawbar  hitch  pin  hole  was  designated  as  the  RVP 
for  these  vehicles.  As  shown  in  Fig.  3,  the  vision  sensor  was  rigidly 
mounted  to  the  chassis  of  the  tractor  with  the  lens  pointed  down¬ 
ward  so  that  the  field  of  view  was  centered  on  the  drawbar 
hitch  pivoting  location  (pin  hole).  To  calibrate  the  vision  sensor, 
a  Cognex®  100-mm  calibration  grid  was  centered  under  the  hitch 
pin  hole  with  the  horizontal  axis  parallel  to  the  rear  axle  of  the 
tractor. 

To  set  an  A-B  line  on  the  northern  side  of  the  track,  the  tractor 
was  manually  operated  close  to  the  expansion  seam  line  at  least  an 
hour  before  the  test.  At  the  beginning  of  each  test  run,  the  tractor 
was  located  at  the  northeast  corner  of  the  track  facing  west  (ready 
to  travel  in  a  counterclockwise  direction  around  the  track).  The  data 
acquisition  system  was  started,  the  tractor  moved  forward,  and  the 
auto-guidance  system  was  engaged.  The  tractor  traveled  along  the 
north  side  of  the  track  with  a  swath  width  of  0  m  with  respect  to  the 
original  A-B  line  until  it  reached  the  end  of  the  northern  pass.  At 


Fig.  2.  Test  pattern  representing:  (a)  a  typical  field  operation  and  (b)  adapted  to  the  oval-shaped  test  track. 
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Fig.  3.  The  permanent  reference  line  tracking  vision  sensor  mounted  to  the  chassis 
of  a  test  tractor. 


the  curve,  the  operator  manually  steered  the  tractor  counterclock¬ 
wise  around  the  western  curve  of  the  track  and  lined  it  up  with  the 
expansion  seam  of  the  southern  portion  of  the  track.  While  enter¬ 
ing  the  southern  pass,  the  auto-guidance  system  was  re-engaged 
with  a  swath  width  of  39.9  m  from  the  original  A-B  line.  The  auto¬ 
guidance  mode  remained  engaged  along  the  southern  pass  of  the 
track  until  the  tractor  reached  the  eastern  end.  The  operator  again 
took  control  and  manually  steered  the  tractor  around  the  curve. 
At  this  point,  depending  on  the  lap  number  and  the  number  of  laps 
required  by  the  test  speed,  the  tractor  either  turned  around  to  travel 
in  a  clockwise  (CW)  direction  or  continued  travel  to  complete  the 
number  of  counterclockwise  laps  required  for  that  travel  speed.  The 
tractor  traveled  at  speeds  ranging  from  0.5  to  5.0  m/s.  To  account 
for  increased  travel  distances  due  to  higher  speed,  specific  numbers 
of  laps  in  the  counterclockwise,  clockwise,  and  counterclockwise 
direction  were  performed  so  the  travel  time  in  one  direction  was 
just  greater  than  7.5  min.  Table  1  shows  the  number  of  laps  in  each 
direction  required  for  each  tested  travel  speed. 

As  the  tractor  entered  into  a  new  straight  pass,  the  auto¬ 
guidance  mode  was  engaged  before  the  end  of  the  curve  unless 
a  change  of  travel  direction  from  clockwise  to  counterclockwise 
was  made.  In  this  scenario,  the  auto-guidance  system  was  engaged 
before  a  set  point  on  the  northern  side  of  the  track.  Data  taken 
before  this  point  were  excluded  from  the  analysis. 

To  process  the  data,  average  valid  relative  position  and  time 
measurements  were  obtained  for  each  1-m  segment  of  the  track. 
The  local  projection  of  geographic  position  measurements  obtained 
using  an  additional  GNSS  receiver  located  above  the  visual  sen¬ 
sor  was  based  on  the  WGS-84  ellipsoid  and  used  to  assign  each 
measurement  to  a  corresponding  test  track  segment.  Relative 
pass-to-pass  XTE  terms  were  found  by  comparing  relative  posi¬ 
tion  measurements  (visual  sensor  output)  between  two  collocated 
points  from  passes  in  a  single  test  run  that  were  obtained  during 


Table  1 

Travel  directions  for  selected  travel  speeds. 


Travel  speed 

Number  of  lapsa 

m/s 

mile/h 

CCW 

CW 

CCW 

0.5b 

1.1 

0.5 

0.5 

0.5 

1.0 

2.2 

1 

1 

1 

2.5 

5.6 

2 

2 

2 

3.0 

6.7 

3 

3 

3 

3.5 

7.8 

3 

3 

3 

4.0 

8.9 

3 

3 

3 

4.5 

10.1 

4 

4 

4 

5.0 

11.2 

4 

4 

4 

a  CCW  and  CW  indicate  counterclockwise  and  clockwise  directions. 
b  When  traveling  at  0.5  m/s,  it  was  impossible  to  make  a  full  lap  around  the  track 
so  only  the  northern  side  of  the  track  was  used. 


Table  2 

Comparisons  for  determination  of  pass-to-pass  auto-guidance  error. 


Travel  speed 

Lap  number3 

m/s 

mile/h 

Pass  1 

Pass  2 

Pass  3 

0.5b 

1.1 

1 

2 

3 

1.0 

2.2 

1 

2 

3 

2.5 

5.6 

1 

4 

5 

2 

3 

6 

3.0 

6.7 

1 

6 

7 

2 

5 

8 

3 

4 

9 

3.5 

7.8 

1 

6 

7 

2 

5 

8 

3 

4 

9 

4.0 

8.9 

1 

6 

7 

2 

5 

8 

3 

4 

9 

4.5 

10.1 

1 

8 

9 

2 

7 

10 

3 

6 

11 

4 

5 

12 

5.0 

11.2 

1 

8 

9 

2 

7 

10 

3 

6 

11 

4 

5 

12 

a  Comparisons  were  made  horizontally  between  lap  numbers  (for  example,  for  a 
speed  of  3  m/s,  the  following  comparisons  were  made:  1-6,  2-5, 3-4,  6-7,  5-8,  and 
4-9). 

b  Comparisons  were  made  for  the  northern  pass  only. 

travel  in  opposite  directions  with  revisit  time  under  15  min.  Long¬ 
term  error  was  also  detected  by  comparing  cross-track  position 
measurements  between  passes  obtained  during  travel  in  oppo¬ 
site  directions  but  from  different  test  runs.  Table  2  shows  lap 
comparisons  for  pass-to-pass  error,  while  Table  3  shows  the  lap 
comparisons  for  long-term  error  determination. 

From  unsigned  values  of  relative  XTE,  cumulative  distributions 
were  constructed  with  95%  errors  identified,  as  a  95%  absolute  error 
is  the  most  frequently  used  value  referred  to  in  promotional  lit¬ 
erature.  Mean  values  of  signed  XTE  were  calculated  to  determine 
bias  in  the  auto-guidance  system.  Unfortunately,  because  of  the 
potential  to  obtain  non-zero  mean  of  a  signed  error  distribution 
caused  by  either  temporal  positioning  offset  or  non-symmetric 
response  of  steering  controller,  it  is  not  appropriate  to  conduct  con¬ 
ventional  variance  analysis  to  compare  results  from  different  tests. 
Therefore,  the  mean  and  the  standard  deviation  of  the  normally 
distributed  signed  errors  were  used  to  numerically  (5,000  random 
points)  assess  the  probability  of  absolute  errors  being  greater  for 
one  test  with  respect  to  another  test.  For  illustration,  every  reported 
test  result  has  been  compared  to  a  frequently  cited  by  equipment 
manufacturers  claim  of  95%  error  within  25.4  mm  (1  in.).  Generally, 
such  a  claim  should  mean  a  normal  error  distribution  with  zero 
mean  and  12.7  mm  standard  deviation,  which  has  been  denoted  as 
a  benchmark  claim. 

Variations  in  pass  entrance  errors  caused  differences  in  the 
lengths  of  transition  to  steady-state  operation,  so  two  different  sets 
of  data  with  transition  regions  of  different  lengths  were  analyzed. 
These  were  labeled  as  inner  and  outer  marks  (refer  to  Fig.  1).  The 
inner  marks  were  located  30.5  m  (100  ft)  inside  the  outer  marks  to 
allow  extra  distance  for  the  auto-guidance  system  to  reach  equi¬ 
librium  before  entering  the  region  where  the  data  were  considered 
valid  for  the  purpose  of  testing.  While  the  inner  marks  allowed 
a  longer  distance  for  the  auto-guidance  steering  control  to  reach 
steady-state  operation,  the  outer  marks  provided  a  more  complete 
revisit  time  distribution  as  more  data  points  were  considered  part 
of  the  valid  dataset. 
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Table  3 

Comparisons  for  determination  of  long-term  auto-guidance  error. 


Travel  speed 

Test  run  number  comparisons  (lap  comparisons) 

m/s 

mile/h 

1-2 

2-1 

1-3 

3-1 

2-3 

3-2 

0.5  and  1.0 

1.1  and  2.2 

l,l-2,2a 

2, 1-1, 2 

1,1-3, 2 

3, 1-1, 2 

2, 1-3, 2 

3, 1-2, 2 

1,2-2, 3 

2, 2-1, 3 

1,2-3, 3 

3, 2-1, 3 

2, 2-3, 3 

3, 2-2, 3 

2.5 

5.6 

1,1-2, 4 

2, 1-1, 4 

1,1-3, 4 

3, 1-1, 4 

2, 1-3, 4 

3, 1-2, 4 

1,2-2, 3 

2, 2-1, 3 

1,2-3, 3 

3, 2-1, 3 

2, 2-3, 3 

3, 2-2, 3 

1,4-2, 5 

2, 4-1, 5 

1,4-3, 5 

3, 4-1, 5 

2, 4-3, 5 

3, 4-2, 5 

1,3-2, 6 

2, 3-1, 6 

1,3-3, 6 

3, 3-1, 6 

2, 3-3, 6 

3, 3-2, 6 

3.0,  3.5  and  4.0 

6.7,  7.8  and  8.9 

1,1-2, 6 

2, 1-1, 6 

1,1-3, 6 

3, 1-1, 6 

2, 1-3, 6 

3, 1-2, 6 

1,2-2, 5 

2, 2-1, 5 

1,2-3, 5 

3, 2-1, 5 

2, 2-3, 5 

3, 2-2, 5 

1,3-2, 4 

2, 3-1, 4 

1,3-3, 4 

3, 3-1, 4 

2, 3-3, 4 

3, 3-2, 4 

1,6-2, 7 

2, 6-1, 7 

1,6-3, 7 

3, 6-1, 7 

2, 6-3, 7 

3, 6-2, 7 

1.5-2, 8 

2, 5-1, 8 

1,5-3, 8 

3, 5-1, 8 

2, 5-3, 8 

3, 5-2, 8 

1,4-2, 9 

2, 4-1, 9 

1,4-3, 9 

3, 4-1, 9 

2, 4-3, 9 

3, 4-2, 9 

4.5  and  5.0 

10.1  and  11.2 

1,1-2, 8 

2, 1-1, 8 

1,1-3, 8 

3, 1-1, 8 

2, 1-3, 8 

3, 1-2, 8 

1,2-2, 7 

2, 2-1, 7 

1,2-3, 7 

3, 2-1, 7 

2, 2-3, 7 

3, 2-2, 7 

1,3-2, 6 

2, 3-1, 6 

1,3-3, 6 

3, 3-1, 6 

2, 3-3, 6 

3, 3-2, 6 

1,4-2, 5 

2, 4-1, 5 

1,4-3, 5 

3, 4-1, 5 

2, 4-3, 5 

3, 4-2, 5 

1,8-2, 9 

2, 8-1, 9 

1,8-3, 9 

3, 8-1, 9 

2, 8-3, 9 

3, 8-2, 9 

1,7-2,10 

2,7-1,10 

1,7-3,10 

3,7-1,10 

2,7-3,10 

3,7-2,10 

1,6-2,11 

2,6-1,11 

1,6-3,11 

3,6-1,11 

2,6-3,11 

3,6-2,11 

1,5-2,12 

2,5-1,12 

1,5-3,12 

3,5-1,12 

2,5-3,12 

3,5-2,12 

a  X,Y  format:  X  is  the  test  run  number  and  Y  is  the  lap  number. 


2.3.  Test  system  evaluation  (Test  A ) 

For  Test  A,  the  three  selected  travel  speeds  were:  1.0,  2.5,  and 
5.0  m/s.  A  John  Deere  8520  tractor  with  a  Trimble  AgGPS  RTK 
Autopilot™  system  (Nav  2  controller)  was  provided  by  Trimble 
Navigation  Limited  (Sunnyvale,  CA).  The  tractor  was  outfitted  with 
the  vision  sensor  above  the  RVP,  as  well  as  with  an  additional  geo- 
referencing  receiver  (Outbacks,  Outback  Guidance,  Hiawatha,  KS). 
The  test  served  as  the  initial  trial  of  what  could  become  a  standard¬ 
ized  test  protocol.  After  data  processing,  a  follow-up  test  (Test  B) 
to  investigate  the  travel  speed  effect  was  conducted. 

2.4.  Travel  speed  testing  (Test  B ) 

Many  field  operations  are  done  with  travel  at  the  speed  of 
approximately  2.5  m/s  (5.6mile/h),  but  some  operations  (e.g., 
spraying)  may  require  travel  speeds  of  approximately  5.0  m/s 
(11.2mile/h)  or  higher.  Specialty  crops  may  require  travel  speeds 
as  low  as  0.1  m/s  (0.22  mile/h).  Unfortunately,  the  tractor  available 
for  this  second  test  was  not  capable  of  operating  in  auto-guidance 
mode  at  0.1  m/s.  Therefore,  to  observe  the  effect  of  travel  speed  on 
auto-guidance  error,  eight  speed  settings  were  selected  for  Test  B: 
0.5,  1.0,  2.5,  3.0,  3.5,  4.0,  4.5,  and  5.0  m/s.  For  this  testing,  a  UNL- 
owned  John  Deere  7820  tractor  equipped  with  a  Trimble  AgGPS 
RTK  Autopilot™  system  (Nav  1  controller)  and  an  AgLeader  Insight 
(AgLeader  Technology,  Inc.,  Ames,  IA)  terminal  were  used.  The  same 
test  scheme  was  used,  with  the  tractor  traveling  a  fixed  number  of 
laps  around  the  track  at  a  fixed  travel  speed,  as  shown  in  Fig.  2.  How¬ 
ever,  the  0.5  m/s  travel  speed  did  not  allow  completion  of  a  full  lap 
around  the  track,  so  only  the  northern  side  of  the  track  was  used 
instead.  For  Test  B,  the  georeferencing  receiver  was  an  RTK-level 
Trimble  AgGPS  442  (GPS/GLONASS). 

3.  Results  and  discussion 

3.1.  Test  system  evaluation  (Test  A) 

Shown  in  Fig.  4  are  the  revisit  time  distributions  for  the  inner 
and  outer  marks  that  indicated  a  moderately  uniform  distribu¬ 


tion  of  the  revisit  time  intervals  between  0  and  15  min.  Variations 
in  the  number  of  occurrences  of  revisit  time  intervals  were  due 
to  the  effect  of  distance-based  segmentation  of  the  test  track. 
Higher  travel  speeds  had  more  revisit  occurrences  as  compared 
to  slower  travel  speeds.  Although  segmentation  of  the  data  on 
a  time  basis  might  be  a  more  appropriate  approach  in  terms  of 
revisit  time  distributions,  a  distance-based  segmentation  approach 
had  greater  practical  value  since  it  involves  the  potential  for 
skips  and  overlaps  that  can  occur  when  operating  under  field 
conditions. 

During  the  analysis  process,  as  noted  above,  greater  errors  were 
observed  at  the  beginning  of  each  northern  or  southern  pass,  caused 
by  the  variation  in  manual  steering  to  align  the  tractor  with  the 
track  line  before  engaging  the  auto-guidance  mode  of  operation. 
Therefore,  the  inner  and  outer  marks  were  used  to  run  separate 
analyses.  Although  the  system  was  engaged  in  auto-guidance  mode 
before  crossing  the  outer  marks,  using  the  inner  marks  to  des¬ 
ignate  valid  data  collection  areas  allowed  for  a  longer  steering 
stabilization  distance.  For  example,  Fig.  5  illustrates  the  cross¬ 
track  position  measurements  around  the  track  at  a  high  speed 
(5.0  m/s).  Significantly  higher  differences  in  relative  tractor  posi¬ 
tion  between  passes  in  the  opposite  direction  occurred  at  the 
entrances  to  the  northern  and  southern  portions  of  the  track. 
At  slower  travel  speeds  (1.0  and  2.5  m/s)  these  differences  were 
negligible. 

Fig.  6  illustrates  the  cumulative  distributions  of  unsigned  pass- 
to-pass  and  long-term  error  terms  found  using  both  inner  and  outer 
marks.  Table  4  summarizes  the  statistics  of  signed  error  distribu¬ 
tions  corresponding  to  Test  A.  From  the  summary,  it  appears  the 
average  signed  errors  were  only  a  few  millimeters,  except  for  the 
long-term  error  when  operating  at  5.0  m/s  and  using  the  outer 
marks  to  process  the  data  (26  mm).  This  provides  a  foundation  for 
assuming  that  the  auto-guidance  system  did  not  have  a  steering 
bias  and  the  error  followed  a  normal  distribution  centered  near 
zero.  The  exception  for  the  high  speed  was  caused  by  the  difference 
in  the  manual  entrance  methods  for  the  northern  or  southern  test 
pass  during  different  test  runs.  It  appears  the  auto-guidance  con¬ 
troller  converged  more  slowly  to  a  straight  line  at  the  beginning  of 
the  test,  which  signifies  some  differences  in  terms  of  the  timing  of 
system  engagement  and  angle  of  entering  the  straight  passes.  On 
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Fig.  4.  Revisit  time  distributions  for  1.0,  2.5,  and  5.0  m/s  travel  speeds  with  inner  and  outer  data  filtering  marks  (Test  A). 
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Fig.  5.  Cross-track  relative  position  measurements  around  the  track  (5.0  m/s,  Test  A). 
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Fig.  6.  Cumulative  distributions  of  unsigned  (a  and  c)  pass-to-pass,  and  (b  and  d)  long-term  auto-guidance  error  when  using  (a  and  b)  inner,  and  (c  and  d)  outer  data  filtering 
marks  (Test  A). 


Table  4 

Summary  of  Test  A  results. 


Test  speed  (m/s) 

Error  type 

End  marks 

Signed  error  (mm) 

Probability  that  unsigned  error  is 
greater  than  the  benchmark  claim3 

Mean 

St.  dev. 

1.0 

Pass-to-pass 

Inner 

0.6 

11.7 

42% 

Outer 

0.6 

10.7 

40% 

Long-term 

Inner 

0.1 

11.4 

43% 

Outer 

0.2 

10.5 

38% 

2.5 

Pass-to-pass 

Inner 

-1.4 

8.8 

34% 

Outer 

-1.0 

11.1 

41% 

Long-term 

Inner 

-0.2 

9.6 

36% 

Outer 

-1.7 

11.5 

42% 

5.0 

Pass-to-pass 

Inner 

-3.5 

20.0 

60% 

Outer 

-3.0 

32.4 

74% 

Long-term 

Inner 

-5.1 

21.4 

65% 

Outer 

25.9 

21.1 

80% 

a  Signed  error  distribution  with  zero  mean  and  12.7  mm  (0.5  in.)  standard  deviation. 
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Table  5 

Summary  of  Test  B  results. 


Test  speed  (m/s) 

Error  type 

End  marks 

Signed  error  (mm) 

Probability  that  unsigned  error  is 
greater  than  the  benchmark  claim3 

Mean 

St.  dev. 

0.5 

Pass-to-pass 

Inner 

1.8 

23.6 

66% 

Outer 

3.0 

23.6 

66% 

Long-term 

Inner 

-0.3 

22.2 

65% 

Outer 

-2.1 

21.0 

62% 

1.0 

Pass-to-pass 

Inner 

-1.0 

27.3 

69% 

Outer 

-0.4 

26.6 

71% 

Long-term 

Inner 

1.7 

27.4 

70% 

Outer 

1.2 

27.1 

71% 

2.5 

Pass-to-pass 

Inner 

-3.2 

22.4 

66% 

Outer 

-2.0 

23.6 

66% 

Long-term 

Inner 

1.3 

26.6 

69% 

Outer 

0.7 

27.3 

70% 

3.0 

Pass-to-pass 

Inner 

3.3 

23.0 

65% 

Outer 

2.9 

23.9 

66% 

Long-term 

Inner 

5.2 

26.7 

70% 

Outer 

6.9 

29.4 

73% 

3.5 

Pass-to-pass 

Inner 

-2.6 

23.4 

66% 

Outer 

-3.7 

24.3 

67% 

Long-term 

Inner 

2.1 

27.2 

69% 

Outer 

2.8 

30.2 

73% 

4.0 

Pass-to-pass 

Inner 

0.0 

28.3 

69% 

Outer 

0.4 

31.9 

75% 

Long-term 

Inner 

-3.1 

29.3 

71% 

Outer 

-3.4 

32.4 

74% 

4.5 

Pass-to-pass 

Inner 

1.3 

29.0 

72% 

Outer 

-0.3 

33.4 

75% 

Long-term 

Inner 

-2.8 

28.9 

71% 

Outer 

-4.6 

32.7 

74% 

5.0 

Pass-to-pass 

Inner 

3.3 

43.5 

81% 

Outer 

2.3 

47.5 

81% 

Long-term 

Inner 

-1.8 

46.1 

82% 

Outer 

-0.4 

49.5 

83% 

3  Signed  error  distribution  with  zero  mean  and  12.7  mm  (0.5  in.)  standard  deviation. 


the  other  hand,  this  bias  was  not  found  for  any  consecutive  test  run 
(pass-to-pass  error)  or  when  using  the  inner  marks. 

Based  on  Test  A  results,  the  cumulative  95%  auto-guidance 
error  values  for  the  lower  travel  speeds  of  1.0  and  2.5  m/s  con¬ 
firmed  the  20-mm  accuracy  claimed  by  the  manufacturer.  The 
results  also  indicated  that  travel  speed  caused  auto-guidance 
error  to  increase  when  operating  at  5.0  m/s,  which  suggested  the 
need  for  follow-up  testing.  These  increases  resulted  in  60-80% 
probability  of  the  errors  being  higher  than  the  established  bench¬ 
mark,  whereas  50%  probability  means  no  difference  at  all.  Also, 
it  was  noted  that  the  probability  was  less  than  50%  for  1  and 
2.5  m/s  travel  speed,  which  means  that  the  guidance  error  was 
lower  than  25.4  mm  (1  in.)  95%  of  time.  Yet,  none  of  these  differ¬ 
ences  were  found  significant  at  a  =  0.1  (less  than  10  or  more  than 
90%  probability). 

3.2.  Travel  speed  testing  (Test  B ) 

Although  every  effort  was  made  during  this  test  to  assure  the 
same  turning  path  was  followed,  the  effect  of  significant  positioning 
error  in  the  transition  from  manual  to  steady-state  auto-guidance 


steering  operation  at  the  5.0  m/s  travel  speed  was  observed  (as 
during  Test  A).  This  phenomenon  was  once  again  not  significant 
for  the  slow  and  medium  travel  speeds.  The  signed  Test  B  auto¬ 
guidance  error  estimates  are  summarized  in  Table  5.  In  this  case, 
the  mean  signed  errors  were  less  than  their  standard  deviations 
for  all  travel  speeds  (maximum  at  6.9  mm  with  29.4  mm  standard 
deviation  for  long-term  error  estimated  using  outer  marks  when 
operating  at  3  m/s).  This  confirms  the  assumption  there  was  no 
significant  steering  bias  in  the  system. 

However,  as  shown  in  Figs.  7  and  8,  the  cumulative  distributions 
of  the  unsigned  XTE  terms  indicate  an  increase  of  auto-guidance 
error  values  for  Test  B  in  comparison  to  Test  A.  At  1.0  and  2.5  m/s 
travel  speeds,  the  95%  auto-guidance  error  estimates  increased 
from  approximately  20  mm  to  approximately  50  mm.  For  the 
5.0  m/s  travel  speed,  this  increase  was  from  about  40  mm  to  as 
much  as  90  mm  (125%).  The  probability  that  the  guidance  error 
is  greater  than  the  benchmark  claim  rose  to  62-83%.  This  error 
increase  over  Test  A  could  be  due  to  multiple  factors:  (1 )  differences 
in  the  auto-guidance  systems;  (2)  the  quality  of  the  positioning  sig¬ 
nal;  (3)  the  system  set-up  and  vehicle  dynamics.  Test  A  involved 
a  larger  tractor  with  the  most  up-to-date  control  hardware  and 
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Fig.  7.  Cumulative  distributions  of  auto-guidance  error  when  using  inner  data  filtering  marks  for:  (a)  slow,  (b)  medium,  and  (c)  fast  travel  speeds  (Test  B). 


software,  while  Test  B  was  performed  with  an  older  version  of 
the  control  hardware.  The  Test  A  system  was  operated  and  fine- 
tuned  by  company  representatives  according  to  the  procedures 
available  to  the  consumer  but  usually  set  once  and  not  readjusted 


thereafter.  The  Test  B  tractor  was  obtained  from  an  actual  produc¬ 
tion  farm,  fitted  with  the  measurement  system  and  tested  without 
the  recommended  calibration  procedure.  Despite  these  differ¬ 
ences,  for  the  medium  travel  speed  tests  processed  using  inner 
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Fig.  8.  Cumulative  distributions  of  auto-guidance  error  when  using  outer  data  filtering  marks  for:  (a)  slow,  (b)  medium,  and  (c)  fast  travel  speeds  (Test  B). 


marks,  the  estimated  95%  error  measurements  were  still  below 
50  mm. 

Again,  it  was  noted  that  the  inner  marks  provided  lower 
95%  error  estimates  compared  to  the  outer  marks.  The  dif¬ 


ference  between  pass-to-pass  and  long-term  auto-guidance 
error  terms  was  below  10  mm  for  most  travel  speeds, 
which  confirms  the  long-term  repeatability  of  the  RTK-level 
GNSS. 
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Fig.  9.  Relationship  between  travel  speed  and  auto-guidance  error  when  using  (a)  inner  and  (b)  outer  data  filtering  marks. 


Table  6 

Pass-to-pass  error  comparison3. 


Combination  II 

Combination  I 

Test  A 

Test  B 

1.0  m/s 

2.5  m/s 

5.0  m/s 

0.5.  m/s 

1.0  m/s 

2.5  m/s 

3.0  m/s 

3.5  m/s 

4.0  m/s 

4.5  m/s 

5.0  m/s 

Test  A 

1.0  m/s 

50% 

35% 

63% 

68% 

71% 

65% 

67% 

68% 

71% 

73% 

80% 

2.5  m/s 

65% 

50% 

72% 

73% 

78% 

73% 

74% 

73% 

78% 

78% 

86% 

5.0  m/s 

37% 

28% 

50% 

51% 

57% 

51% 

52% 

52% 

58% 

59% 

71% 

Test  B 

0.5  m/s 

32% 

27% 

49% 

50% 

52% 

47% 

48% 

47% 

53% 

55% 

68% 

l.Om/s 

29% 

22% 

43% 

48% 

50% 

42% 

43% 

43% 

50% 

50% 

63% 

2.5  m/s 

35% 

27% 

49% 

53% 

58% 

50% 

50% 

49% 

55% 

57% 

68% 

3.0  m/s 

33% 

26% 

48% 

52% 

57% 

50% 

50% 

49% 

54% 

55% 

68% 

3.5  m/s 

32% 

27% 

48% 

53% 

57% 

51% 

51% 

50% 

55% 

54% 

66% 

4.0  m/s 

29% 

22% 

42% 

47% 

50% 

45% 

46% 

45% 

50% 

49% 

62% 

4.5  m/s 

27% 

22% 

41% 

45% 

50% 

43% 

45% 

46% 

51% 

50% 

60% 

5.0  m/s 

20% 

14% 

29% 

32% 

37% 

32% 

32% 

34% 

38% 

40% 

50% 

a  Probability  that  an  individual  unsigned  pass-to-pass  error  measurement  (inner  marks)  representing  test  and  travel  speed  Combination  I  is  greater  than  such  a  measurement 
representing  Combination  II. 


Fig.  9  illustrates  the  relationship  between  travel  speed  and  esti¬ 
mated  95%  auto-guidance  error  for  both  tests.  Table  6  summarizes 
levels  of  probability  that  an  individual  error  measurement  for  each 
combination  of  auto-guidance  systems  and  travel  speeds  is  greater 
than  an  individual  measurement  for  any  other  such  combination 
(pass-to-pass  errors  and  inner  marks  only).  It  appeared  the  great¬ 
est  increase  occurred  when  switching  from  the  travel  speed  of  4.5 
to  5.0  m/s.  This  increase  could  be  due  to  vehicle  dynamics  and  the 
need  for  a  longer  distance  to  enter  the  steady-state  operation  mode. 
At  slow  travel  speeds  (0.5  and  1 .0  m/s)  the  performance  of  the  auto¬ 
guidance  system  did  not  appear  to  be  different  from  performance 
with  a  travel  speed  of  2.5  m/s.  Therefore,  it  seems  reasonable  to 
focus  on  two  or  three  travel  speeds  when  developing  the  stan¬ 
dardized  test  procedure.  For  instance,  the  standardized  test  could 
include  a  medium  (2.5  m/s)  and  a  fast  (5.0  m/s)  travel  speed,  for 
which  auto-guidance  error  was  found  to  increase  50-100%.  Lower 
than  1 .0  m/s  speeds  could  be  considered  in  special  cases  when  very 
slow  field  operations  are  anticipated. 

4.  Conclusions 

In  this  study,  a  high-speed  vision  sensor-based  system  was  used 
to  quantify  the  accuracy  of  GNSS-based  auto-guidance  systems. 


Pass-to-pass  and  long-term  cross-track  errors  were  defined  as  the 
ability  of  the  system  to  repeat  the  same  pass  in  an  opposite  direc¬ 
tion  within  short  (15  min)  and  long  (several  hours)  time  periods.  A 
comparison  of  three  different  travel  speeds  (Test  A)  and  eight  differ¬ 
ent  travel  speeds  (Test  B)  revealed  the  ability  of  the  developed  test 
system  to  differentiate  among  operating  conditions  that  may  influ¬ 
ence  the  performance  of  an  auto-guidance  system.  It  was  shown 
that  relatively  high  travel  speeds  resulted  in  a  substantially  higher 
auto-guidance  error  as  compared  to  the  slower  speeds.  Using  the 
RTK-level  receiver,  the  difference  between  pass-to-pass  and  long¬ 
term  performance  was  found  to  be  negligible. 

The  increase  of  up  to  100%  in  the  corresponding  error  estimated 
between  Test  B  and  Test  A  could  have  been  caused  by  a  variety  of 
factors  including  differences  in  tractor  and  auto-guidance  systems. 
However,  only  high  travel  speeds  caused  auto-guidance  errors  to 
increase  greater  than  50  mm.  This  indicates  that  a  standardized  test 
procedure  should  include  at  least  two  specified  test  travel  speeds 
(e.g.,  2.5  and  5.0  m/s).  Neither  test  showed  any  difference  between 
slow  and  medium  travel  speeds.  Therefore,  testing  with  slow  travel 
speeds  should  be  considered  only  for  special  circumstances.  Finally, 
it  was  noted  that  using  the  outer  marks  did  not  allow  sufficient 
space  for  the  system  to  reach  steady-state  operation  when  traveling 
at  the  high  speeds. 
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